Guided image upsampling using bitmap tracing

ABSTRACT

A method of increasing resolution of an image includes generating vector contours associated with a first image and scaling the vector contours to a second resolution. The first image has a first resolution, and the second resolution is greater than the first resolution. The vector contours are rendered to generate a guiding image at the second resolution, and a second image is generated from the first image based on the guiding image, by using joint upsampling. The second image is generated at the second resolution. The vector contours can be generated by bitmap tracing binary images at different quantization levels. An apparatus and computer readable device implementing the method of increasing the resolution of an image are also provided.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/734,636, filed on Jun. 9, 2015, which is a continuation of U.S.patent application Ser. No. 14/025,350, filed on Sep. 12, 2013, whichissued as U.S. Pat. No. 9,076,236 on Jul. 7, 2015, the disclosures ofwhich are incorporated by reference herein in their entireties.

BACKGROUND

Field

The present disclosure relates to upsampling of images and, morespecifically, to guided image upsampling of an image that increasesresolution.

Related Art

Image upsampling, or generation of an image at a greater, i.e., higherresolution than an initial image, is a fundamental image processingproblem. The need for generating a higher resolution image from aninitial image is rapidly increasing, in part, because of the rate atwhich content is being created, and because of the enhanced resolutionof display devices, including the latest smartphones, tablets, andhigh-definition televisions. There is a particular need for increasingthe resolution of images for display on such devices from traditionallylow resolution images, such as images sent via email, downloaded fromthe Internet, or streamed from a content provider.

SUMMARY

Features of the disclosure will become apparent from the followingdetailed description considered in conjunction with the accompanyingdrawings. It is to be understood, however, that the drawings aredesigned as an illustration only and not as a definition of the limitsof this disclosure.

The disclosure is directed to a method of increasing resolution of animage. The method includes receiving, using a processing device, a firstimage, which has a first resolution, and generating, using theprocessing device, vector contours associated with a first image. Themethod further includes scaling, using the processing device, the vectorcontours to a second resolution, where the second resolution is greaterthan the first resolution; and rendering, using the processing device,the vector contours to generate a guiding image at the secondresolution. Using the processing device, a second image is thengenerated, at the second resolution, from the first image based on theguiding image.

In one aspect, the vector contours are generated by generating twobinary images associated with the first image, where each of the twobinary images is based on a different quantization level; and bitmaptracing each of the two binary images to convert the two binary imagesto the vector contours.

In another aspect, the second image is generated by interpolating thefirst image to the second resolution based on the guiding image, wherethe interpolating includes using joint upsampling to guide interpolatingto the second resolution based on the guiding image.

In yet another aspect, wherein the first image is a color image, themethod further includes converting the color image to a grayscale imagein response to receiving the first image. The vector contours associatedwith the first image are generated from the grayscale image.

In an additional aspect, the first image is an n-bit grayscale image,and the vector contours are generated by generating a number N, which isless than 2^(n), of binary images and bitmap tracing each of the binaryimages to generate the vector contours. Each of the binary images isbased on a different quantization level.

In yet another aspect, the method further includes generating ahistogram of pixel values of the first image and allocating thedifferent quantization levels based on low discrepancy sequence samplingof the histogram.

The vector contours are preferably rendered from an outermostquantization level L_(N) to innermost quantization level L₁.

The disclosure is also related to a computer-readable device to storeinstructions that, when executed by a processing device, cause theprocessing device to perform operations to increase resolution of animage. The operations include receiving a first image, where the firstimage has a first resolution; generating vector contours associated withthe first image; and scaling the vector contours to a second resolution,where the second resolution is greater than the first resolution. Theoperations further include rendering the vector contours to generate aguiding image at the second resolution; and generating a second imagefrom the first image based on the guiding image, the second image beinggenerated at the second resolution.

In one aspect, the operations for generating vector contours furtherinclude generating two binary images associated with the first image andbitmap tracing each of the two binary images to convert the two binaryimages to the vector contours. Each of the two binary images is based ona different quantization level.

In another aspect, generating the second image includes interpolatingthe first image to the second resolution based on the guiding image, theinterpolating including using joint upsampling to guide interpolating tothe second resolution based on the guiding image.

In a further aspect, the first image is a color image. The operationsfurther include converting the color image to a grayscale image inresponse to receiving the first image, the vector contours associatedwith the first image being generated from the grayscale image.

In yet another aspect, the first image is an n-bit grayscale image. Theoperations include generating a number N, which is less than 2^(n), ofbinary images and bitmap tracing each of the binary images to generatethe vector contours. Each of the binary images is based on a differentquantization level.

In one aspect, the operations for rendering the vector contours furtherinclude rendering the vector contours from an outermost quantizationlevel L_(N) to an innermost quantization level L₁.

The present disclosure is also directed to an apparatus to increaseresolution of an image. The apparatus includes a processing device; andmemory to store instructions that, when executed by the processingdevice, cause the processing device to perform operations includingreceiving a first image, where the first image has a first resolution;and generating vector contours associated with the first image. Theoperations further include scaling the vector contours to a secondresolution, where the second resolution is greater than the firstresolution; rendering the vector contours to generate a guiding image atthe second resolution; and generating a second image from the firstimage based on the guiding image. The second image is generated at thesecond resolution.

In one aspect, the operations for generating vector contours furtherinclude generating two binary images associated with the first image andbitmap tracing each of the two binary images to convert the two binaryimages to the vector contours. Each of the two binary images is based ona different quantization level.

In another aspect, the operations for generating the second imagefurther include interpolating the first image to the second resolutionbased on the guiding image, the interpolating including using jointupsampling to guide interpolating to the second resolution based on theguiding image.

In still another aspect, the apparatus includes a display device, theoperations further including displaying the second image on the displaydevice.

In yet another aspect, the first image is an n-bit grayscale image. Theoperations include generating a number N, which is less than 2^(n), ofbinary images and bitmap tracing each of the binary images to generatethe vector contours. Each of the binary images is based on a differentquantization level.

In an additional aspect, the operations further include generating ahistogram of pixel values of the first image and allocating thedifferent quantization levels based on low discrepancy sequence samplingof the histogram.

In one aspect, the operations for rendering the vector contours furtherinclude rendering the vector contours from an outermost quantizationlevel L_(N) to an innermost quantization level L₁.

Other features of the present disclosure will become apparent from thefollowing detailed description considered in conjunction with theaccompanying drawings. It is to be understood, however, that thedrawings are designed as an illustration only and not as a definition ofthe limits of the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawings constitute a part of this disclosure and include examples,which may be implemented in various forms. It is to be understood thatin some instances, various aspects of the disclosure may be shownexaggerated or enlarged to facilitate understanding. The teaching of thedisclosure can be readily understood by considering the followingdetailed description in conjunction with the accompanying drawings.

FIG. 1 is a schematic representation of an embodiment of a method ofincreasing resolution of an image in accordance with the presentdisclosure.

FIG. 2 is a schematic representation of another embodiment of a methodof increasing resolution of an image in accordance with the presentdisclosure.

FIG. 3 is a block diagram representation of a device interfaced with anexemplary machine in the form of a computing system configured toperform methods according to one or more embodiments.

FIG. 4 is a block diagram showing at least a portion of an exemplarymachine in the form of a computing system configured to perform methodsaccording to one or more embodiments.

DETAILED DESCRIPTION

Upsampling is achieved by convolving an initial (lower resolution) imagewith an interpolation kernel or filter, and resampling the result on atarget (higher resolution) grid. Interpolation filters, such as bilinearand bicubic filters, for use in such upsampling processes arewell-known. Such methods include setting the value of a pixel in thetarget image at a higher resolution (“HR”) as a weighted average of thevalues in the neighborhood of the pixel in an initial lower resolution(“LR”) image. While the same filter is applied across the entire imagefor computational efficiency, these methods generate unsatisfactoryartifacts in many cases. These artifacts are most perceptible alongsharp edges within the image content, and can take the form of blurring,aliasing, ringing, and blocking.

To address these shortcomings, many new adaptive image interpolationalgorithms have been developed, which can generally be classified intothree different categories: edge-guided; example-based or“super-resolution” algorithms; and “other.” The majority of knownmethods are edge-guided. Most edge-guided methods are based oninterpolating the missing higher resolution pixels in the target (HR)image using estimated local covariance in the target image from thelocal covariance coefficients in the initial LR image.

Example-based super-resolution algorithms typically utilize a databaseto look-up or learn correspondences between the initial lower resolutionimage and target image patches. These databases can be constructed aheadof time from large image collections, but have more recently beenconstructed on the fly using only the initial input image, or only asmall portion of it.

Additional methods that fall outside the edge-based and example-basedcategories include game-theoretic approaches to upsampling, whichgenerally incorporate an iterative feedback-control loop. While thistechnique can lead to high quality results, performance degradation canoccur when the number of iterations of the loop increases.

Another image filter is the well-known “bilateral filter,” which is anedge-preserving non-linear filter that uses both a spatial (or domain)filter kernel and a range filter kernel evaluated on the data valuesthemselves. It has been used primarily in denoising and tone mappingapplications. In particular, the rationale behind the bilateral filteris that for a pixel to influence another pixel, it should not onlyoccupy a nearby spatial location but should also have a similar colorintensity value. More formally, the bilateral filtered result for apixel p is:

$\begin{matrix}{{{{BF}\lbrack I\rbrack}_{p} = {\frac{1}{W_{p}}{\sum\limits_{q \in S}^{\;}{I_{q}{G_{\sigma \; s}\left( {{p - q}} \right)}{G_{\sigma \; r}\left( {{I_{p} - I_{q}}} \right)}}}}},} & (1)\end{matrix}$

where I is the input image, G_(σs) and G_(σr), which are typically 2DGaussian kernels, represent a spatial filter and range filter,respectively, and S (in q ∈+S) denotes the neighborhood of interest inthe spatial domain. W_(p) is a normalization factor that ensures thatthe pixel weights sum to one.

One variant of the bilateral filter is the Joint Bilateral Filter(“JBF”). In the JBF, a second so-called “guiding image” (Ĩ), which is ofthe same resolution as the input image, is used to define the edges thatmust be preserved:

$\begin{matrix}{{{JBF}\lbrack I\rbrack}_{p} = {\frac{1}{W_{p}}{\sum\limits_{q \in S}^{\;}{I_{q}{G_{\sigma \; s}\left( {{p - q}} \right)}{{G_{\sigma \; r}\left( {{{\overset{\sim}{I}}_{p} - {\overset{\sim}{I}}_{q}}} \right)}.}}}}} & (2)\end{matrix}$

While straight-forward implementations of the bilateral filter and itsvariants are computationally expensive, there have been attempts tosignificantly improve running times by approximating the filterequation. For example, a guided filter that is based on a local linearmodel and that provides a faster and superior edge-preserving filter isdisclosed in K. He, et al., “Guided image filtering,” IEEE Trans.Pattern Analysis Machine Intelligence (Accepted), vol. 99, no.PrePrints, pp. 1-14, 2012 (referred to herein as “He”), the entirety ofwhich is incorporated herein by reference.

The methodology of the Joint Bilateral Filter has been successfullyapplied to a particular upsampling problem that arises from the need toreduce memory costs associated with performing computationally intensiveprocessing tasks, such as stereo depth, image colorization, and tonemapping. In particular, an input image is first downsampled to a lowerresolution in order to perform the necessary image processing tasks atcomputationally manageable levels. After processing, the low resolutionsolution must then be upsampled to the original resolution of the inputimage.

One technique for upsampling the low resolution solution, which is basedon the Joint Bilateral Filter, is disclosed in Kopf, J., et al., “JointBilateral Upsampling,” ACM Transactions on Graphics, Vol. 26, No. 3, pp.96-100 (2007) (referred to herein as “Kopf”), the entirety of which isincorporated herein by reference. Given a high-resolution guiding imageI, and a low resolution solution L obtained after processing an inputimage, Kopf discloses that an upsampled solution can be obtained byapplying a “Joint Bilateral Upsampling” (referred to also as “JBU”)technique as follows:

$\begin{matrix}{{{JBU}\left\lbrack \overset{\sim}{L} \right\rbrack}_{p} = {\frac{1}{W_{p}}{\sum\limits_{q_{\downarrow} \in S}^{\;}{L_{q_{\downarrow}}{G_{\sigma \; s}\left( {{p_{\downarrow} - q_{\downarrow}}} \right)}{G_{\sigma \; r}\left( {{I_{p} - I_{q}}} \right)}}}}} & (3)\end{matrix}$

where p and q denote integer coordinates of the pixels in I, and p ↓ andq ↓ denote the corresponding coordinates in the low resolution solutionL. The guiding image I for interpolating to the upsampled resolution inthe JBU technique disclosed in the Kopf reference is the original inputimage. Accordingly, unlike the application of a Joint Bilateral Filteras shown in equation (2), the JBU of equation (3) operates at twodifferent resolutions: the high resolution of the guiding image and lowresolution of the downsampled low resolution solution L.

The present disclosure is directed to a method, apparatus, andcomputer-readable device to increase resolution of an image. Although asequence of images, for example, video images can be upsampled inaccordance with the present disclosure, the image of interest ispreferably a still image and can be a computer-generated image orgenerated by any appropriate electronic device, such as an imagecapturing/display device capable of capturing and transmitting and/ordisplaying an image or a sequence of images, including adigital/electronic still camera, video camera, image scanner,smartphone, tablet, television, and so on.

The method of the present disclosure can be implemented within adedicated computing system, such as the computer system 100 shown inFIG. 4. Referring also to FIG. 3, the computer system 100 can receivethe image for increasing its resolution directly from any imagecapturing/display device 52 via a device interface 54 over a bus 110,for example. In various embodiments, the device interface 54 can be anyknown wired or wireless interface (such as USB, IEEE standardinterfaces, Bluetooth™ and so on) capable of transmitting the capturedimage(s) to the system 100 for implementing the methods of the presentdisclosure to increase image resolution. The transmitted images can bestored in any machine readable medium, as described herein, associatedwith the system 100 for processing in accordance with the methods of thedisclosure.

The methods of the present disclosure can also be implemented within adedicated computer system, such as the system 100 shown in FIG. 4, toincrease the resolution of an image that is stored on anycomputer-readable media, including any removable computer-readablemedium.

In addition, although the methods of the present disclosure aredescribed as implemented in an embodiment of the computer system 100shown in FIG. 4, it is understood that in additional embodiments, themethods can also be implemented within any suitable imagecapturing/display device 60, including a digital/electronic stillcamera, video camera, image scanner, smartphone, tablet, televisionsmartphones, tablets, and televisions, having analogous processing andstorage devices as computer system 100 for storing and executinginstructions. The device 60 preferably also includes a display 56 fordisplaying the upsampled image.

A “bitmap” as used herein and as conventionally known is a binary image,which can be represented as a sampled grid of black and white pixels.

“Bitmap tracing” is the process of converting a bitmap (binary image) toa list of vector contours, for example, Bezier curves, or to a vectoroutline, which describes the binary image as an algebraic expression ofits vector contours.

“Rendering” is a process of converting a vector outline to a bitmap, andis the converse of bitmap tracing.

“Joint Upsampling” as used herein is any operation for generating ahigher resolution image from an input (lower resolution) image under theguidance of another (guiding) image at a higher resolution.

Joint Upsampling includes, for example, but is not limited to, the JointBilateral Upsampling (“JBU”) technique disclosed in the Kopf reference,and the guided filter joint upsampling disclosed in the He reference.

A “guiding image” as used herein is an image used to guide theinterpolation of an input image to a greater resolution in an upsamplingprocess.

In the present disclosure, a guiding image is proposed that can be used,for joint upsampling an input (lower resolution) image, for example, inaccordance with a Joint Bilateral Upsampling algorithm, such as shown inequation (3) supra, and variations thereof, for increasing theresolution of an image, rather than of a low resolution solution.Referring to FIG. 1, given an initial (first) image having a firstresolution, one embodiment of a method 10 of the present disclosure forincreasing resolution of the first image 12, includes generating vectorcontours 14 associated with the first image 12, and then scaling thevector contours 16 to a second (increased) resolution, where the secondresolution is greater than the first resolution. The scaled vectorcontours are then rendered to generate a guiding image at the secondresolution 18. A second image at the second (greater) resolution canthen be generated by interpolating the first image to the second(greater) resolution using the guiding image 20.

The guiding image of the present disclosure, which can be a grayscaleimage, advantageously preserves the image content structure of theinitial, lower resolution, image sufficiently to avoid the usualartifacts (blurring, jaggies, and the like) associated with filter-basedupsampling.

Referring to FIG. 2, in one embodiment 30, the vector contoursassociated with the initial image are generated by bitmap tracing abinary representation of the initial image. In accordance withembodiments of the present disclosure, the bitmap traced image can beupsampled in a lossless manner to the desired resolution and then usedas a guide for upsampling the original image content. Unlike some othermethods, there are no power-of-two restrictions on the scaling factorsfor upsampling an image, since the vector contours generated can bescaled to any size without loss of quality.

As is known in the art, a binary image can also be represented as avector outline, typically using spline approximations, which describesthe image as an algebraic expression of its vector contours, e.g.,Bezier curves. One advantage of representing an image as a vectoroutline, or as vector contours, is that the vector contours can bescaled to any size without loss of quality. Vector outlines areespecially popular in the description of fonts and logos.

The process of converting bitmaps into vector outlines is known asbitmap tracing. Several methods are known in the art for bitmap tracing.One popular open-source tracing tool which can be applied to the methodsof the present disclosure is “Potrace.”

The Potrace algorithm (the name is a derivation of “polygon tracer”)transforms a bitmap into a vector outline in several steps. Initially,the bitmap is decomposed into a number of paths, which form theboundaries between black and white areas. In the second step, each pathis approximated by an optimal polygon which is then transformed into asmooth outline. Optionally, the resulting curve is optimized by joiningconsecutive Bezier curve segments together wherever possible. Furtherdetails of the algorithm are disclosed in P. Selinger,“Potrace:apolygon-based tracing algorithm,”http://potrace.sourceforge.net/potrace.pdf (Sep. 20, 2003),(referred toherein as “Selinger”), which is incorporated in its entirety herein byreference. Once the image is converted to vector contours 14 by bitmaptracing, for example, the vector contours can be scaled to the desiredsize (resolution) 16. The scaled contours are then rendered to producethe guiding image 18.

It has been shown that the rendered output of a traced image generatedusing Potrace's algorithm exhibits superior edge structure compared toother interpolated images (scaled by the same factor) produced usingsimple interpolation filters (such as the bicubic or bilinear filter).

Referring again to FIG. 2, if an initial image 32 for upsampling is incolor, it is preferably converted to grayscale 34 before generating theguiding image.

In particular embodiments, a grayscale image I, which is either theinitial image 32, or the converted grayscale image 35 of an initialcolor image, is converted to a list of vector contours given Nquantization levels, L₁<L₂<L₃<. . . <L_(N). The following table, Table1, provides an example of pseudo-code that describes a conversion of thegrayscale image to vector contours by bitmap tracing.

TABLE 1 Binarize Grayscale image and bitmap trace to vector form Input:Grayscale image I; N quantization levels L₁ < L₂ < L₃ < ... < L_(N)Output: List of vector contours V  Initialize V = Ø  for i = 1 → N do   Initialize binary bitmap B to a white image    for each pixel p in Bdo      if I[p] < L_(i) then       B[p] = 0; // make binary pixel levelblack      end if    end for    List of contours C = Potrace(B)   Associate all contours in C with quantization level L_(i)    Append Cto V  end for  return V

As shown in Table 1, two or more (N) binary images B can be generated atdifferent quantization levels L 36 as a sequence of black and whiteimages. Each binary image is then bitmap traced to generate a list ofvector contours 38 for each quantization level, using Potrace, forexample. For any n-bit image, the list of vector contours can begenerated for a number N<2^(n) quantization levels. For 8-bit grayscaleimages, for example, a list of vector contours can be generated for upto 255 quantization levels, which are not necessarily equally-spaced.However, experimentation has shown that for most (natural) images, 64 or128 equally-spaced levels are sufficient to produce very accurate andvisually indistinguishable results.

In one embodiment, a number N of quantization levels is chosen to beequally spaced. For example, for an n-bit grayscale image, a numberN<2^(n) quantization levels L₁, L₁, . . . , L_(N) are chosen to bespread uniformly across the quantization space for bitmap tracing asfollows:

${L_{i} = {{{{INT}\left\lbrack {\frac{i - 1}{N - 1}*\left( {2^{n} - 1} \right)} \right\rbrack}\mspace{14mu} {for}\mspace{14mu} i} = 1}},\ldots \mspace{14mu},{N.}$

In other embodiments, the quantization levels are allocated based on ahistogram of the grayscale input image. The image histogram provides thedistribution of the 2^(n) intensity values from which the quantizationlevels can be sampled, either randomly or quasi-randomly, for example,using low discrepancy sequences.

A low-discrepancy sequence, as well-known in the art, is a sequence withthe property that for all values of N, its subsequence x₁, . . . , x_(N)has a low discrepancy. The discrepancy of a sequence is considered lowif the proportion of points in the sequence falling into an arbitraryset A is close to proportional to the measure of A, as may happen, forexample, in the case of an equidistributed sequence. Specificdefinitions of discrepancy differ regarding the choice of A and how thediscrepancy for every A is computed and combined.

Low discrepancy sequences offer some advantages over random sampling interms of coverage and in the small sample size regime, and there areseveral open source libraries like LDSquences(sourceforge.net/projects/ldsequences) and libseq(www.multires.caltech.edu/software/libseq) to generate such sequences.

Once the list of vector contours is generated, they can be scaled to thedesired increased resolution 40 without significant loss in quality. Asdescribed in Selinger, the bitmap tracing produces closed contoursseparating one color from another. Each of the closed contours in thescaled vector map are then rendered 42 to generate the guiding image 43using any appropriate method known in the art, preferably from outermostto innermost quantization level (L_(N) to L₁). Appropriate renderingsoftware is available, for example, through the open-source librsvg(https://live.gnome.org/librsvg) and cairo(http://http://www.cairographics.org) libraries.

The guiding image 43 is then used to perform Joint Upsampling of theinitial image 32, which is in grayscale or color, to generate a higherresolution image. In one embodiment, the guided filter joint upsamplingtechnique disclosed in the He reference is applied, using the guidingimage 43 of the present disclosure.

It should be noted that although the vector contours are preserved inthe guiding image, the discrete nature of the quantization processappears as banding artifacts, which worsen as the scaling of theresolution increases. Accordingly, the guiding image itself does notprovide a high quality upsampled output image. On the other hand, theseartifacts do not affect the filtering in a Joint Upsampling algorithmsignificantly because the quantization levels, using Potrace, forexample, are close.

FIG. 4 is a block diagram of an embodiment of a machine in the form of acomputing system 100, within which a set of instructions 102, that whenexecuted, may cause the machine to perform any one or more of themethodologies disclosed herein. In some embodiments, the machineoperates as a standalone device. In some embodiments, the machine may beconnected (e.g., using a network) to other machines. In a networkedimplementation, the machine may operate in the capacity of a server or aclient user machine in a server-client user network environment. Themachine may comprise a server computer, a client user computer, apersonal computer (PC), a tablet PC, a personal digital assistant (PDA),a cellular telephone, a mobile device, a palmtop computer, a laptopcomputer, a desktop computer, a communication device, a personal trusteddevice, a web appliance, a network router, a switch or bridge, or anymachine capable of executing a set of instructions (sequential orotherwise) that specify actions to be taken by that machine.

The computing system 100 may include a processing device(s) 104 (e.g., acentral processing unit (CPU), a graphics processing unit (GPU), orboth), program memory device(s) 106, and data memory device(s) 108,which communicate with each other via a bus 110. The computing system100 may further include display device(s) 112 (e.g., liquid crystalsdisplay (LCD), a flat panel, a solid state display, or a cathode raytube (CRT)). The computing system 100 may include input device(s) 146(e.g., a keyboard), cursor control device(s) 116 (e.g., a mouse), diskdrive unit(s) 118, signal generation device(s) 119 (e.g., a speaker orremote control), and network interface device(s) 124.

The disk drive unit(s) 118 may include machine-readable medium(s) 120,on which is stored one or more sets of instructions 102 (e.g., software)embodying any one or more of the methodologies or functions disclosedherein, including those methods illustrated herein. The instructions 81may also reside, completely or at least partially, within the programmemory device(s) 106, the data memory device(s) 108, and/or within theprocessing device(s) 104 during execution thereof by the computingsystem 100. The program memory device(s) 106 and the processingdevice(s) 104 may also constitute machine-readable media. Dedicatedhardware implementations, not limited to application specific integratedcircuits, programmable logic arrays, and other hardware devices canlikewise be constructed to implement the methods described herein.Applications that may include the apparatus and systems of variousembodiments broadly include a variety of electronic and computersystems. Some embodiments implement functions in two or more specificinterconnected hardware modules or devices with related control and datasignals communicated between and through the modules, or as portions ofan application-specific integrated circuit. Thus, the example system isapplicable to software, firmware, and hardware implementations.

In accordance with various embodiments of the present disclosure, themethods described herein are intended for operation as software programsrunning on a computer processor. Furthermore, software implementationscan include, but are not limited to, distributed processing orcomponent/object distributed processing, parallel processing, or virtualmachine processing constructed to implement the methods describedherein.

One embodiment contemplates a machine-readable medium orcomputer-readable device containing instructions 102, or that receivesand executes instructions 102 from a propagated signal so that a deviceconnected to a network environment 122 can send or receive voice, video,images or data, and communicate over the network 122 using theinstructions 102. The instructions 102 may further be transmitted orreceived over a network 122 via the network interface device(s) 124. Themachine-readable medium may also contain a data structure for storingdata useful in providing a functional relationship between the data anda machine or computer in an illustrative embodiment of the disclosedsystems and methods.

While the machine-readable medium 120 is shown in an example embodimentto be a single medium, the term “machine-readable medium” should betaken to include a single medium or multiple media (e.g., a centralizedor distributed database, and/or associated caches and servers) thatstore the one or more sets of instructions. The term “machine-readablemedium” shall also be taken to include any medium that is capable ofstoring, encoding, or carrying a set of instructions for execution bythe machine and that cause the machine to perform any one or more of themethodologies of the present embodiment. The term “machine-readablemedium” shall accordingly be taken to include, but not be limited to:solid-state memories such as a memory card or other package that housesone or more read-only (non-volatile) memories, random access memories,or other re-writable (volatile) memories; magneto-optical or opticalmedium such as a disk or tape; and/or a digital file attachment toe-mail or other self-contained information archive or set of archives,which may be considered a distribution medium equivalent to a tangiblestorage medium. Accordingly, the embodiment is considered to include anyone or more of a tangible machine-readable medium or a tangibledistribution medium, as listed herein and including art-recognizedequivalents and successor media, in which the software implementationsherein are stored.

In one non-limiting, example embodiment, the computer-readable devicecan include a solid-state memory, such as a memory card or other packagethat houses one or more non-volatile read-only memories. Further, thecomputer-readable device can be a random access memory or other volatilere-writable memory. Additionally, the computer-readable device caninclude a magneto-optical or optical medium, such as a disk or tapes orother storage device to capture carrier wave signals such as a signalcommunicated over a transmission medium. A digital file attachment to ane-mail or other self-contained information archive or set of archivesmay be considered a distribution medium that is equivalent to a tangiblestorage medium. Accordingly, the disclosure is considered to include anyone or more of a computer-readable device or a distribution medium andother equivalents and successor media, in which data or instructions maybe stored.

In accordance with various embodiments, the methods, functions or logicdescribed herein may be implemented as one or more software programsrunning on a computer processor. Dedicated hardware implementationsincluding, but not limited to, application specific integrated circuits,programmable logic arrays and other hardware devices can likewise beconstructed to implement the methods described herein. Furthermore,alternative software implementations including, but not limited to,distributed processing or component/object distributed processing,parallel processing, or virtual machine processing can also beconstructed to implement the methods, functions or logic describedherein.

It should also be noted that software which implements the disclosedmethods, functions or logic may optionally be stored on a tangiblestorage medium, such as: a magnetic medium, such as a disk or tape; amagneto-optical or optical medium, such as a disk; or a solid statemedium, such as a memory card or other package that houses one or moreread-only (non-volatile) memories, random access memories, or otherre-writable (volatile) memories. A digital file attachment to e-mail orother self-contained information archive or set of archives isconsidered a distribution medium equivalent to a tangible storagemedium. Accordingly, the disclosure is considered to include a tangiblestorage medium or distribution medium as listed herein, and otherequivalents and successor media, in which the software implementationsherein may be stored.

EXAMPLE

A guiding image formed in accordance with the present disclosure wasused in the guided filter joint upsampling technique disclosed in the Hereference, using the following weighting factors for spatial and rangefilters: σs=2 and σr=0.015. The algorithm was implemented as asingle-threaded CPU application using C++ on a Linux workstation (withtwo Intel® e5506 CPUs @ 2.13 GHz). Although the multiple CPU or GPUcores available were not leveraged, the algorithm does lend itself toparallel processing and the running times could be further reduced bymodifying the implementation. Currently, the implementation takes up toroughly 1.5 seconds to scale 200×200 pixel images by a factor of 2. Todemonstrate how the algorithm scales, the upsampling factor was alsovaried from 2 to 8 for a variety of images and the running timesmonitored. It was found that the vast majority of the time (95%) wasspent on computing the high resolution guiding image in the tracingalgorithm, with the remaining time spent in the upampling step. Thetracing algorithm was shown to be clearly dependent upon the number ofvector contours generated for the quantization levels, or in otherwords, the edge complexity of the initial low resolution image content.This was evident as some images with the same pixel size required moretime than others.

The quality of upsampled images generated in accordance with the presentdisclosure were also found to produce sharp images that are devoid ofthe blocky artifacts exhibited by upsampled images produced from thesame initial image using bicubic filter techniques. The upsampled imagesgenerated in accordance with the present disclosure also did not exhibitthe noise, discretization or pixelization along the edges that appearedin the bicubic-filtered images. In addition, unlike some otherupsampling methods known in the art, no training sets are required. Asingle input image is upsampled as desired.

Although peak signal-to-noise ratio (PSNR) is often used toquantitatively compare upsampling results, it is known that PSNR is notalways an accurate measure of visual quality of natural images. It isnot expected that the joint upsampling algorithm would necessarilyproduce improved PSNR numbers because the tracing algorithm inherentlydisplaces the edges slightly in the image. This directly impacts imagequality measurements such as PSNR. Nevertheless, the present upsamplingalgorithm was compared to bicubic interpolation on a collection ofimages from the Kodak Lossless True Color Image Suite(http://r0k.us/graphics/kodak) and found to exhibit, on average, PSNRvalues 0.3 dB lower than those exhibited using bicubic interpolation.

It should be noted that although the present specification may describecomponents and functions implemented in the embodiments with referenceto particular standards and protocols, the disclosed embodiments are notlimited to such standards and protocols.

Although specific example embodiments have been described, it will beevident that various modifications and changes may be made to theseembodiments without departing from the broader scope of the inventivesubject matter described herein. Accordingly, the specification anddrawings are to be regarded in an illustrative rather than a restrictivesense. The accompanying drawings that form a part hereof, show by way ofillustration, and not of limitation, specific embodiments in which thesubject matter may be practiced. The embodiments illustrated aredescribed in sufficient detail to enable those skilled in the art topractice the teachings disclosed herein. Other embodiments may beutilized and derived therefrom, such that structural and logicalsubstitutions and changes may be made without departing from the scopeof this disclosure. This Detailed Description, therefore, is not to betaken in a limiting sense, and the scope of various embodiments isdefined only by the appended claims, along with the full range ofequivalents to which such claims are entitled.

Such embodiments of the inventive subject matter may be referred toherein, individually and/or collectively, by the term “embodiment”merely for convenience and without intending to voluntarily limit thescope of this application to any single embodiment or inventive conceptif more than one is in fact disclosed. Thus, although specificembodiments have been illustrated and described herein, it should beappreciated that any arrangement calculated to achieve the same purposemay be substituted for the specific embodiments shown. This disclosureis intended to cover any and all adaptations or variations of variousembodiments. Combinations of the above embodiments, and otherembodiments not specifically described herein, will be apparent to thoseof skill in the art upon reviewing the above description.

In the foregoing description of the embodiments, various features aregrouped together in a single embodiment for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting that the claimed embodiments have more features than areexpressly recited in each claim. Rather, as the following claimsreflect, inventive subject matter lies in less than all features of asingle disclosed embodiment. Thus the following claims are herebyincorporated into the Detailed Description, with each claim standing onits own as a separate example embodiment.

The Abstract is provided to comply with 31 C.F.R. § 1. 12(b), whichrequires an abstract that will allow the reader to quickly ascertain thenature of the technical disclosure. It is submitted with theunderstanding that it will not be used to interpret or limit the scopeor meaning of the claims.

What is claimed is:
 1. A method of increasing image resolution, themethod comprising: scaling, using an imaging device, vector contoursassociated with a first image based on a second resolution, the firstimage comprising a first resolution, the second resolution being greaterthan the first resolution; generating, using the imaging device, aguiding image based on the vector contours, the guiding image beinggenerated based on the second resolution; and generating, using theimaging device, a second image based on the first image using theguiding image, the second image being generated based on the secondresolution.
 2. The method defined by claim 1, wherein the guiding imageis a grayscale image.
 3. The method defined by claim 1, furthercomprising: generating binary images associated with the first image,the binary images comprising different quantization levels; andconverting the binary images to the vector contours by bitmap tracingthe binary images.
 4. The method defined by claim 3, further comprisingrepresenting the binary images as a vector outline using splineapproximation.
 5. The method defined by claim 1, wherein generating thesecond image further comprises interpolating the first image to thesecond resolution using the guiding image and joint up sampling.
 6. Themethod defined by claim 1, wherein the first image is an n-bit grayscaleimage, the method further comprising: generating N binary images; andgenerating the vector contours by bitmap tracing the binary images, Nbeing less than 2^(n), the binary images being based on differentquantization levels.
 7. The method defined by claim 6, wherein thedifferent quantization levels are unequally spaced apart.
 8. The methoddefined by claim 6, further comprising: generating a histogram of pixelvalues associated with the first image; and allocating the differentquantization levels based on low-discrepancy sequence sampling of thehistogram.
 9. The method defined by claim 8, wherein the low-discrepancysequence sampling further comprises quasi-randomly sampling thedifferent quantization levels.
 10. A computer-readable device storinginstructions that, when executed by an imaging device, cause the imagingdevice to perform operations comprising: scaling vector contoursassociated with a first image based on a second resolution, the firstimage comprising a first resolution, the second resolution being greaterthan the first resolution; generating a guiding image based on thevector contours, the guiding image being generated based on the secondresolution; and generating a second image based on the first image usingthe guiding image, the second image being generated based on the secondresolution.
 11. The computer-readable device defined by claim 10,wherein the guiding image is a grayscale image.
 12. Thecomputer-readable device defined by claim 10, wherein the operationsfurther comprise: generating binary images associated with the firstimage, the binary images comprising different quantization levels; andconverting the binary images to the vector contours by bitmap tracingthe binary images.
 13. The computer-readable device defined by claim 12,wherein the operations further comprise representing the binary imagesas a vector outline using spline approximation.
 14. Thecomputer-readable device defined by claim 10, wherein the operationsfurther comprise interpolating the first image to the second resolutionusing the guiding image and joint upsampling.
 15. The computer-readabledevice defined by claim 10, wherein the first image is an n-bitgrayscale image, the operations further comprising: generating N binaryimages; and generating the vector contours by bitmap tracing the binaryimages, N being less than 2, the binary images being based on differentquantization levels.
 16. The computer-readable device defined by claim15, wherein the different quantization levels are unequally spacedapart.
 17. The computer-readable device defined by claim 15, theoperations further comprising: generating a histogram of pixel valuesassociated with the first image; and allocating the differentquantization levels based on low-discrepancy sequence sampling of thehistogram.
 18. The computer-readable device defined by claim 17, whereinthe low-discrepancy sequence sampling further comprises quasi-randomlysampling the different quantization levels.
 19. An apparatus thatincreases image resolution, the apparatus comprising: an imaging device;and a memory device storing instructions that, when executed by theimaging device, cause the imaging to perform operations comprising:scaling vector contours associated with a first image based on a secondresolution, the first image comprising a first resolution, the secondresolution being greater than the first resolution; generating a guidingimage based on the vector contours, the guiding image being generatedbased on the second resolution; and generating a second image based onthe first image using the guiding image, the second image beinggenerated based on the second resolution.
 20. The apparatus thatincreases image resolution defined by claim 19, wherein the guidingimage is a grayscale image.